Parallelizing and Vectorizing Compilers

Programming computers is a complex and constantly evolving activity. Technological innovations have consistently increased the speed and complexity of computers, making programming them more difficult. To ease the programming burden, computer languages have been developed. A programming language employs syntactic forms that express the concepts of the language. Programming language concepts have evolved over time, as well. At first the concepts were centered on the computing machine being used. Gradually, the concepts of computer languages have evolved away from the machine, toward problem-solving abstractions. Computer languages require translators, called compilers, to translate programs into a form that the computing machine can use directly, called machine language. The part of a computer that does arithmetical or logical operations is called the processor. Processors execute instructions, that determine the operation to perform. An instruction that does arithmetic on one or two numbers at a time is called a scalar instruction. An instruction that operates on a larger number of values at once (e.g. 32 or 64) is called a vector instruction. A processor that contains no vector instructions is called a scalar processor and one that contains vector instructions is called a vector processor. If the machine has more than one processor of either type, it is called a multiprocessor or a parallel computer. The first computers were actually programmed by connecting components with wires. This was a tedious task and very error prone. If an error in the wiring was made, the programmer had to visually inspect the wiring to find the wrong connection. The innovation of stored-program computers eliminated the physical wiring job. The stored-program computer contained a memory device that could store a series of binary digits (1s and 0s), a processing device that could carry out several operations, and devices for communicating to the outside world (input/output devices). An instruction for this kind of computer consisted of a numeric code indicating which operation to carry out, as well as an indication of the operands to which the operation was applied. Debugging such a machine language program involved inspecting memory and finding the binary digits that were loaded incorrectly. This could be tedious and time-consuming. Then another innovation was introduced symbolic assembly language and a assembler program to translate assembly language programs into machine language. Programs could be written using names for memory locations and registers. This eased the programming burden considerably, but people still had to manage a group of registers, had to check status bits within the processor, and keep track of other machine-related hardware. Though the problems changed somewhat, debugging was still tedious and time-consuming. A new pair of innovations was required to lessen the onerous attention to machine details a new language (Fortran), closer to the domain of the problems being solved, and a program for translating Fortran programs into machine code, a Fortran compiler. The 1954 Preliminary Report on Fortran stated that “. . . Fortran should virtually eliminate coding and debugging . . . ”. That prediction, unfortunately, did not come true, but the Fortran compiler was an undeniable step forward, in that it eliminated much complexity from the programming process. The programmer was freed from managing the way machine registers were employed and many other details of the machine architecture. The authors of the first Fortran compiler recognized that in order for Fortran to be accepted by programmers, the code generated by the Fortran compiler had to be nearly as efficient as code written in assembly language. This required sophisticated analysis of the program, and optimizations to avoid unnecessary operations. Programs written for execution by a single processor are referred to as serial, or sequential programs. When the quest for increased speed produced com-